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Executive Summary 

INTRODUCTION 



T his monograph is intended as a guide to support self- 

evaluation of truancy reduction and prevention programs 
by those involved in their ongoing operations. The benefits of 
understanding how to use evaluation methods and information 
to realistically assess the value of truancy prevention programs, 
and the routine integration of evaluation into the operation 
of programs, will lead to more effective practices and greater 
benefits to young people. 

Truancy behavior is multifaceted with an array of contrib- 
uting risk factors and considerable ambiguity about which 
factors are in play in general situations or in individual cases. 
Therefore, effective interventions to reduce truancy tend to 
be similarly complex. The complexity of truancy interven- 
tion programs has a number of implications for the planning 
and practice of evaluation activities. Evaluating the processes 
and effects of such programs requires thoughtful consideration 
of the various service providers involved and the interac- 
tions among them, their respective contributions to program 
activities, specific beneficiaries to be served, intended program 
effects, and measurable or observable criteria defining success- 
ful program implementation and effectiveness in achieving 
objectives and goals. 

Why Evaluate? 

P rogram evaluations are designed to both support and 

demonstrate program effectiveness. Evaluation determines 
truancy program effectiveness through various assessments 
addressing program goals and objectives and the strategies and 
activities aimed at accomplishing them. The key concept in 
evaluation is “value.” The two most important outcomes of 
evaluation are to demonstrate with clarity and validity the 
value of a program in its context and to understand how the 
program produces that value for beneficiaries in the program’s 
context. Evaluation research employs an array of social and or- 
ganizational science research techniques to assess the value of 
the chosen methods in implementing programs and to measure 
the value of program activities in achieving their targeted ser- 
vices to beneficiaries. Evaluation enables programs to periodi- 
cally review their goals and measure their progress 
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toward achieving them. Evaluation measures progress toward 
goal achievement and informs efforts to refine programs to 
increase their effectiveness in particular situations. 

The most important use of evaluation is to guide program 
improvement. Evaluation facilitates informed decisions about 
improving the quality of program activities. Evaluation can sug- 
gest ways to refine or alter programs or services after measuring 
the success of implemented activities. Evidence-based under- 
standing of factors contributing to program success supports a 
program’s potential to serve as a model for effective replication 
or adaptation and increases the potential for dissemination of 
an effective practice to numerous other environments with at- 
tendant expansion of benefits for those the program is intended 
to serve. 

A program can build ownership among stakeholders when 
everyone who is involved in evaluating a program becomes 
invested in its success. Evaluation also enables stakeholders to 
improve their communication and overall effectiveness and use 
evaluation results in organizing and structuring their activities 
to refine cross-agency strategies. 

The processes of evaluation must be adapted to particular 
program goals, objectives, issues, and contexts. This complex of 
factors can be effectively addressed through a logical sequence 
of identifiable stages or phases from initial planning through 
final reporting: 

• Identification of program components, elements, proce- 
dures, and activities to be evaluated 

• Generation of evaluation questions 

• Selection of evaluation design and procedures for data 
collection and analyses 

• Instrumentation 

• Data collection, analysis, summarization, and 
interpretation 

• Preparation of the evaluation report 

Promoting self-evaluation of programs by practitioners is a 
primary goal of this monograph. Although evaluation is based 
on research activities, it is not exclusively an activity for evalu- 
ation specialists. Evaluation is a populist rather than a special- 
ist practice — aiming to produce practical information in the 
service of human beings and amenable to being done by almost 
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any program operator who cares to make it an integral part of 
their project management, as it should be. Essential program 
evaluation activities are readily accessible to program planners 
and operators. This monograph contains tools to assist program 



sultS'based truancy reduction programs; creating effective truancy 
reduction programs including guidelines and worksheets; and 
developing an action and evaluation plan for such programs. 



n effective truancy reduction program — just as any program 



designed to promote systemic change to improve the condi- 
tions of schools, youth, families, and communities — is dependent 
on careful attention to three stages: planning, implementation, 
and evaluation. Evaluation should be an integral part of the 
effort from the beginning. Evaluation data can be used to guide 
program improvement by measuring the program’s progress in 
achieving its goals and can be used to refine or alter programs or 
services to maximize program value in serving its beneficiaries. 

Strategies to reduce truancy require a major investment of 
resources and energy. Eunding agencies, policymakers, and other 
stakeholders want evidence that their investments are producing 
better outcomes for youth, families, and communities. Evalua- 
tion results can account for how resources are used and document 
program effectiveness to justify the costs of such efforts. Evalu- 
ation provides evidence to funding agencies, policymakers, and 
other stakeholders that their investments are producing better 
outcomes for youth, families, and communities. 



operators in developing a Logic Model for the evaluation of re- 



CONCLUSION 
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INTRODUCTION 



T here is a familiar habit of mind that wants to simplify 
complex problems to a minimum possible set of com- 
ponents. Practitioners who work with issues negatively 
affecting youth often hear questions such as, “What is the one 
thing (or the single most important thing) that leads to . . . (truancy, 
dropout, bullying, etc.).” The corollary question is also often 
asked, “What is the one thing (or the single most important thing) 
that can be done to reduce . . . (truancy, dropout, bullying, 
etc.).” However, experience demonstrates that negative out- 
comes for youth, such as truancy, are multifaceted with an array 
of contributing risk factors and considerable ambiguity about 
what factors are in play in general situations or in individual 
cases. Therefore, effective interventions to reduce negative 
outcomes for youth tend to be similarly complex. Since truancy 
is a behavior with complex patterns of ingredients affecting dif- 
ferent individuals in different ways, truancy reduction programs 
typically address a range of issues leading students to skip school. 
Baker, Sigmon, and Nugent (2001, p. 3) observe: 

The [truancy] models that show the most promise, not only of 
reducing truancy, but also of affecting its risk factors, include 
several key components: 

• Parental involvement 

• Meaningful sanctions or consequences for truancy 

• Meaningful incentives for school attendance 

• Ongoing school-based truancy reduction programs 

• Involvement of community resources 
(e.g., law enforcement) 

Truancy reduction programs may involve a variety of services 
including law enforcement through police and family courts, 
mental health services, social services, health services, and 
services of other community-based agencies and organizations 
as well as schools themselves. Program components may include 
surveillance and reporting of chronically truant students, in- 
creasing family involvement in schools and in student behavior 
management, individual counseling, mentoring, academic sup- 
port, or any number of activities targeting interacting issues that 
have been identified as risk factors contributing to truancy in a 
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particular situation. A few descriptions of typical multifaceted 
truancy reduction programs will be illustrative: 

The Louisville Truancy Program seeks to remove or resolve 
all family barriers to regular school attendance and academic 
success. A community team which may include a judicial offi- 
cer, school personnel, a social service provider, a drug/alcohol 
treatment provider, and a mental health counselor, is created 
to address these barriers through regular contact with the fam- 
ily and celebration of all successes. Case managers maintain 
contact with the families on a regular basis and court process 
is held weekly, at the school. 

Project Respect uses family support practices to meet the 
academic and mental health needs of students facing truancy, 
suspension, or expulsion. Community agencies collaborate 
with schools to improve student achievement, coordinate 
community services, and promote healthy and safe learning 
environments. Community advocates work in elementary, 
middle and high schools encouraging parent participation 
in schools and coordinating family activities designed to 
improve family communication and functioning. 

Placer County Peer Court has developed a truancy program 
with a local high school where students who previously had 
difficulty with attendance but have corrected their problem 
now sit as a panel hearing truancy citations issued by the high 
school. After three cuts or unexcused absences, the assistant 
principal issues a citation to the offending student. Judg- 
ments include mediation with teachers, community service, 
drug testing, parenting classes for the parents of the truant 
students, essays, confining the student to campus during lunch 
hours, taking the student’s driver’s license until grades and 
attendance improve, joining school or community activities, 
and tutoring. 

The YWCA Tates Creek Elementary Truancy Prevention 
Program incorporates concepts of delinquency prevention 
by increasing school attachment and decreasing academic 
failure. Major program components are home visits, parent- 
ing and life skills, daily monitoring of attendance, monitoring 
of school grades, additional academic assistance during the 
school day and after school, family activities, and after-school 
activities. 



Truancy Prevention in Action 7 



(Source: All program descriptions above were abstracted from 
the Model Program database of the National Dropout Prevention 
Center Web site, http://www.dropoutprevention.org.) 

The complexity of truancy intervention programs has a 
number of implications for the planning and practice of program 
evaluation activities. Evaluating the processes and effects of 
such multifaceted programs requires thoughtful consideration of 
the various service providers in a program and the interactions 
among them, their respective contributions to program activi- 
ties, specific beneficiaries to be served, intended program effects, 
and measurable or observable criteria defining successful program 
implementation and effectiveness in achieving program objec- 
tives and goals. 
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Why Evaluate? 



T he planning and execution of program evaluation is 
grounded in the reasons why such programs should be 
evaluated. Truancy program evaluations are designed to 
both support and demonstrate program effectiveness. Evalua- 
tions strengthen program design by clearly articulating shared 
goals and objectives. The data collected through evaluation fa- 
cilitates making informed decisions about improving the quality 
of and making constructive changes in the program. Program 
evaluations also help identify and celebrate successes when 
desired outcomes are achieved (Yap, Aldersebaes, Railsback, 
Shaughnessy, & Speth, 2000). 

The most important use of evaluation data is to guide pro- 
gram improvement. Evaluations enable programs to periodically 
review their goals and measure their progress toward achieving 
them and to suggest ways to refine or alter programs or services 
after measuring the success of the activities implemented. 
Evaluation builds accountability. Strategies to reduce truancy 
require a major investment of monetary, human, and material 
resources; time; and energy. Funding agencies, policymakers, 
and other stakeholders want evidence that their investments 
are producing better outcomes for youth, families, and com- 
munities. Evaluation results should account for how resources 
are used and document program effectiveness to justify the costs 
of the effort. A program can build ownership among stakehold- 
ers when everyone who is involved in evaluating a program 
becomes invested in its success. Evaluation also enables the 
collaborative of stakeholders to improve their communication and 
their overall effectiveness and use evaluation results in organizing 
and structuring their activities to refine cross-agency strategies 
(Putting the Pieces Together, 2004). 

Program evaluation is not intended to be, and should not 
be, a fault-finding practice. The key concept in evaluation is 
“value.” Evaluation research employs an array of social and 
organizational science research techniques to assess the value 
of program methods and implementations in promoting goal 
achievement and to measure the value of programs in achieving 
their targeted purposes. A guidebook from the North Central 
Regional Educational Laboratory (Putting the Pieces Together, 
2004) suggests that evaluation does not need to be a dreaded, 
complicated, or negative activity. The evaluation process can 
highlight a program’s strengths and weaknesses and help orga- 
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nizations think creatively about new linkages when evaluation is 
viewed in the following light: 

1. As an activity that is done by and with families and other 
collaborative partners [stakeholders], not to them; 

2. As a continuous process of gathering information about a 
program to periodically assess its progress toward shared 
goals — a tool for measuring how strategies are working and 
whether they are achieving desired results; 

3. As a method for using information to modify and improve 
strategies; and 

4. As a means of improving accountability to partners, chil- 
dren, families, communities, and funding agencies. 

In certain circumstances, properly designed evaluation 
programs can go beyond the immediate assessment of a particu- 
lar program to the larger purpose of contributing to knowledge 
development and the improvement of practice. For instance. 

The Colorado Foundation for Families and Children (2005) is 
conducting evaluations of the Truancy Reduction Demonstra- 
tion Programs for the Office of Juvenile Justice and Delinquency 
Prevention. The main purpose of these evaluations is to document 
the development of community-based programs across the country 
and to identify effective strategies that other communities can 
adopt in their truancy reduction efforts. A secondary purpose of 
the evaluations is to provide preliminary outcome data to identify 
specific interventions that are most effective in keeping youth in 
school. The Foundation is using a logic model of evaluation that 
defines progress from general goals to more specific strategies, to 
the results that have been achieved. A logic model should guide 
all planning and development activities, the design of evaluation 
methodologies, and cover the following topics: 

• Needs and Capacities: “Why are we doing this?” and 
“What strengths do the youth, families, schools, service 
agencies, and community bring to the program?” 

• Goals: “What do we want to achieve?” 

• Strategies and Activities: “Flow are we going to get there?” 

• Milestones or Benchmarks: “Flow will we know we are on 
the right track?” 

• Outcomes: “What were the results?” 
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The Evaluation Process: 
Program Evaluation logic 

AND DEVELOPMENT PELASES 



E valuation determines truancy program effectiveness 

through various assessments addressing program goals and 
objectives and the strategies and activities aimed at ac- 
complishing them. Whatever the benefits targeted by evaluation 
efforts, the processes of evaluation must be adapted to particular 
program goals, objectives, issues, and contexts. This complex of 
factors can be effectively addressed through a logical sequence of 
developmental evaluation phases. 

Truancy reduction program development itself progresses 
through a sequence of usually overlapping phases. The first steps 
in truancy program planning involve definition of the problems 
to be addressed in a particular setting, determination of goals 
and objectives defining the intended results of the program, 
and the means of achieving them. These program planning and 
design steps are also the first steps in the evaluation of a project. 
Program evaluation progresses through several identifiable stages 
or phases from initial planning through final reporting. Smink 
and Stank ( 1992) proposed a useful model of such evaluation 
phases in connection with dropout prevention programs. This 
model serves as well for defining a sequence of evaluation stages 
for truancy programs and includes the following phases (adapted 
from Smink and Stank, 1992, p. 4). 



• Phase 1 Identification of program components , 

elements, procedures, and activities to be 
evaluated 



• Phase II 

• Phase III 

• Phase IV 

• Phase V 

• Phase VI 



Generation of evaluation questions 

Selection of evaluation design and procedures 
for data collection and analyses 

Instrumentation 

Data collection, analysis, summarization, and 
interpretation 

Preparation of the evaluation report 
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These phases of evaluation reflect an underlying logic of deveh 
opment represented hy the stepwise “basic logic or general pattern 
of reasoning” of evaluation proposed by Scriven and which is 
discussed by Fournier. This developmental logic comprises the 
following processes: 

1. Establishing criteria of merit. On what dimensions must the 
evaluand [thing being evaluated] do well? 

2. Constructing standards . How well should the evaluand 
perform? 

3. Measuring performance and comparing with standards . How 
well did the evaluand perform? 

4. Synthesizing and integrating data into a judgment of merit or 
worth. What is the merit or worth of the evaluand? (Fourni- 
er, 1995, as cited in Rossi, Freeman, & Lipsey, 1999) 

The procedural phase model proposed by Smink and Stank 
cited above will provide a general structure for the following 
discussion of evaluation for truancy reduction programs. 

Phase I: Identification of program 

COMPONENTS, ELEMENTS, PROCEDURES, 
AND ACTIVITIES TO BE EVALUATED 

T he identification of components and activities to be ad- 
dressed in evaluation is based on examination of the goals, 
objectives, and strategies envisioned for a truancy reduction 
program. Smink and Stank (1992) propose the following steps for 
this process: 

1. Review program description, proposal, and materials. 

2. Analyze overall program goals and objectives for each com- 
ponent of the program. 

3. Prepare a list of the elements within each program com- 
ponent about which information is needed [for evaluation 
analyses]. 

4. Submit preliminary list of elements to be evaluated to 
vested interest groups for reaction, expansion and approval. 

5. Have final list of elements approved by [program planners] 
and by vested interest groups involved with the program. 

6. Prepare summary of elements [to be evaluated], (p. 7) 



12 Guidelines eor Evaluating Truancy Reduction Programs 



Developing Program Goals, Objectives, and Strategies 

Since they form the foundation for truancy program evalua- 
tion, the processes of developing appropriate goals, objectives, 
and strategies are critical to the design of all procedural and 
analytical aspects of evaluation. Evaluation designs should relate 
to (1) goals based on the shared vision of the collaborative, (2) 
objectives describing what will be done to achieve the goals, and 
(3) strategies or activities designed to accomplish each objective. 

Goal Definition: Goals state the final outcomes that will be 
accomplished by the truancy program and include the following 
characteristics: 

• They are directly related to the collaborative’s vision state- 
ment. 

• They are realistic and attainable. 

• They are well-defined and clearly written. 

• They are logically related to program objectives. 

The goal definition process brings truancy program planners 
into contact with a “criterion problem” in determining what 
specifications of program outcome can be considered mean- 
ingfully related to causes of truancy as well as to measures of 
program effect. An approach to dealing with this double-barreled 
dilemma is to ground goal selection in consideration of funda- 
mental risk factors that either create motivations to skip school 
or amplify them. Examples of such risk factors include: 

• Chronic academic difficulties 

• Lack of parental or home support for school attendance 

• Crime, bullying, or other threats in the school setting 

• Unsupportive relationships between high-risk students and 
school authorities 

• Attractive nuisances in students’ social fields such as gangs, 
drugs, or even employment 

• Family or social pressures to engage in nonschool atten- 
dance for reasons such as earning money or tending to 
younger siblings 

Program goals conceptualized in terms of their pertinence to 
reducing the impact of such risk factors promoting truancy will 
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have conceptual meaningfulness as valid evaluation criteria rep- 
resenting positive program effects in terms of specified outcomes. 

Objective Statement: In developing objectives it is essential 
to review each goal and create at least one objective for each 
goal. To the extent possible, consider relevant research, literature 
reviews, previous data, trends, and other sources of informa- 
tion on best practices. There are two types of objectives: process 
objectives, which describe how a program will be implemented, 
and outcome objectives , which describe the expected results or 
outcomes. Either type of objective should be stated in measurable 
or at least objectively observable terms. A well-written program 
objective should have the following characteristics: 

• Describes a desired process or outcome rather than a specific 
strategy for accomplishing the process or outcome 

• Is clearly stated in measurable terms that 

• define who will change 

• define what will change 

• define when change will occur (e.g., as a timeline of 
“milestones”) 

• Often defines how much change will occur at interim times 
or at program conclusion when sources of information al- 
low for sufficiently precise projections 

• Is logically related to one or more specific strategies directly 
intended to realize the objective 

Strategy Specification: A strategy describes the activity that 
will be implemented in order to achieve the objective. Planning 
or selecting effective strategies should involve the following: 

• Carefully considering the intent of the objective 

• Reviewing available activities and core services 

• Reviewing related best practices research 

• Determining the personnel and resources needed 

• Identifying the personnel and resources available 

• Determining the amount of time required 

• Determining the amount of time available 

• Considering the appropriateness of potential strategies for 
your participants (e.g., age, gender, race, culture, etc.) 

14 Guidelines eor Evaluating Truancy Reduction Programs 



As examples to demonstrate the relationships among pro- 
gram goals, objectives, and strategies, two objectives and the 
strategies for achieving each of them for a school-based project 
to reduce truancy (the goal) could include the following: 

Objective 1. By midyear, teachers and school administrators 
will demonstrate improved skills for dealing with student 
diversity by passing the end-of-training assessment and by 
receiving positive feedback from students on teacher and 
principal evaluation forms at the end of the year. 

Strategy A. Teachers and school administrators will 
attend eight hours of training on dealing with students 
from different cultural backgrounds, ethnic origins, and 
races. 

Strategy B. Teachers and school administrators will pass 
an end-of-training assessment. 

Strategy C. Students will evaluate teachers and school 
administrators on their attitudes to student diversity. 

Objective 2. By midyear, parents, families, or caregivers will 
show evidence of learning the attendance requirements of 
the school district during a formal interview session. 

Strategy A. Parents, family members, or caregivers will 
attend a two-hour conference session explaining the 
school district’s attendance policy. 

Strategy B. A formal interview will be conducted at the 
end of the conference session to determine if the parents, 
family members, or caregivers have understood the at- 
tendance requirements. 



Phase ll: Generation oe evaluation 

QUESTIONS 

E valuation questions in program evaluation serve the same 
purpose as hypotheses in conventional research design. 
These questions specify and operationalize criteria for determin- 
ing at a targeted level of rigor the evaluation’s results as a basis 
for assessing the program’s effectiveness in pursuing and achiev- 
ing its goals. That is, tied to the goal, objective, and strategy 
specifications, evaluation questions structure the methods of de- 
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termining the program’s value. Rossi, Freeman, and Lipsey (2000, 
p. 81) describe a “good evaluation question” as follows: 

The form evaluation questions should take is shaped by the 
functions they perform. Their principal role is to focus the 
evaluation on those areas of program performance at issue for 
key decisionmakers and stakeholders and to facilitate develop- 
ment of a design for data collection that will provide meaning- 
ful information about how well the program is performing. A 
good evaluation question, therefore, must identify a distinct 
dimension of program performance that is at issue and do so in 
such a way that the quality of the performance can be credibly 
assessed. Such assessment, in turn, requires an accurate de- 
scription of the nature of the performance and some standard 
by which it can be evaluated .... Thus a good evaluation 
question must specify some measurable or observable dimen- 
sion of program performance in reference to the criterion by 
which that performance is to be judged. 

Evaluation questions relate directly to the “criterion problem” 
of determining what can be usefully measured or observed that 
will demonstrate a program’s value in reducing the impact of risk 
factors promoting truancy, the risk factors that drive the selection 
of truancy program goals. Several sample questions will illustrate 
how operational evaluation questions can be clearly related to 
truancy risk factors. 

Example Process Evaluation Questions: 

Did all project partners participate in planning sessions for 
development of interagency communication linkages to maxi- 
mize appropriate services to each truant identified for project 
participation? What obstacles were identified that restricted 
participation, and how were they addressed? 

Were all project partners equipped with adequate resources to 
support their planned contribution to accomplishment of cli- 
ent services? If not, what modifications were made to budgets 
to allow for their full participation? 

Did all truant student mentors attend the entire sequence of 
training sessions provided by family court and school staff? 
How were the nonattendees identified, and what changes were 
made for their full participation in future meetings? 
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Were any school district policies restrictive in the identifica- 
tion of truant students? How were they modified to meet the 
objectives of the truancy project? 

Example Outcome Evaluation Questions: 

Were 90% of students identified as truants during the project 
year assigned to family court-based mentors in lieu of incar- 
ceration in state schools? (This question specifies a quantitative 
criterion and a time frame for evaluation of program operation. 

The question relates to truancy risk due to inadequate adult 
supervision and support and also to alienation risks associated with 
truancy reduction interventions that involve separation of students 
from home and family and from regular school settings.) 

Did 90% of program participants complete and present a per- 
sonal plan for high school completion, graduation, and post- 
high school career training? (This program outcome question 
specifies a quantitative criterion for achievement of a specified goal 
of individual life planning and addresses truancy risks associated 
with low valuing of school and lack of long-range expectations and 
planning by participants.) 

Did 90% of program participants demonstrate at least one 
grade letter improvement in final grades in English and 
Mathematics classes for their current school year compared to 
the baseline year? (This question specifies a quantitative crite- 
rion, time frame, and comparison basis for assessing the impact of 
program interventions on academic performance in critical course 
areas and addresses truancy risk associated with low academic 
achievement.) 

Did student attendance among identified truants decline 75% 
during the project year in comparison with the number of tru- 
ancy events that occurred during the baseline year? (This ques- 
tion specifies a quantitative criterion, time frame, and comparison 
basis for evaluation analysis. This is a final outcome question that 
relates to the total constellation of risk factors pertinent to the ra- 
tionale for the truancy reduction program design and addresses the 
global effect of all interventions by all contributing partners.) 
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Phase III: Selection oe evaluation 

DESIGN AND PROCEDURES EOR DATA 
COLLECTION AND ANALYSES 

Choosing an Evaluation Design 

Many federal grant programs call on educational practitioners 
to use “scientifically-based research” to guide their decisions about 
which interventions to implement. The field of K-12 educa- 
tion contains a vast array of interventions that claim to improve 
educational outcomes. The supporting evidence, however, often 
consists of poorly designed studies. Education has seen many 
interventions introduced that profess to produce dramatic gains 
but ultimately yield little in the way of positive and lasting im- 
provement. The type of evaluation design used is critical to valid 
judgment of the effectiveness of an intervention program. 

There are several factors to consider in choosing an evaluation 
research design. The first factor is the purpose of the evaluation. 
When evaluation is conducted to answer formative (process) 
questions (e.g., how well is the program operating, does it need to 
be refined?), the ability to make a causal link may be less impor- 
tant than when it is conducted for high-stakes, summative (out- 
come) purposes (e.g., has the program met its objectives, should 
it be continued?). The second factor that needs to be considered 
is feasibility. Generally, less rigorous research models are easier to 
implement than more rigorous models. For example, an evalua- 
tion design with random assignment of students to intervention 
and control groups, a primary criterion for controlled experimen- 
tal research, is often not feasible in many real world settings. The 
use of naturally existing comparison groups, while less rigorous, is 
often more feasible in actual practice (Yap, Aldersebaes, Rails- 
back, Shaughnessy, & Speth, 2000) and may be the highest level 
of research group assignment possible in a particular program 
situation. 

A guide produced by the U.S. Department of Education’s Insti- 
tute of Education Sciences, Identifying and Implementing Education- 
al Practices Supported by Rigorous Evidence: A User Friendly Guide 
(2003), seeks to provide educational practitioners with tools to 
distinguish practices supported by rigorous evidence from those 
that are not. The guide describes three evaluation designs popular 
in educational evaluation: pre-post study, comparison group study, 
and randomized controlled group study. 
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Pre-Post Study Designs. Probably the most widely used meth' 
od in education is a pre-post'test design. A pre-post study ex- 
amines whether participants in an intervention improve, stay 
the same, or regress during the course of the intervention, and 
then credits any improvement or regression to the interven- 
tion. For example, a school may want to determine if a parent 
notification system will reduce the number of absences among 
students. Data is collected on the number of student absences 
prior to initiating the notification system and the number of 
absences several months later. The pre-test data is compared 
with the post-test data to determine the effectiveness of the 
notification system. The problem with this type of study is 
that, without reference to a control group, it cannot answer 
whether the participants’ improvement or decline would have 
occurred anyway, even without the intervention. 

Comparison Qroup Study Design. A comparison group study 
design (also known as quasi-experimental design) compares 
outcomes for students who participate in the intervention 
(experimental or treatment group) with outcomes for a non- 
participating group (comparison group) with the assignment 
of students to groups being nonrandomized but based on some 
other grouping process. For example, suppose you want to use 
a comparison-group study to test whether a truancy interven- 
tion program is effective. You would compare the attendance 
of students who participate in the intervention program with 
the performance of a comparison group of students who do 
not participate in program. The comparison group might 
be students in neighboring classrooms or schools that don’t 
use the program. The difference in attendance between the 
intervention and comparison groups following the interven- 
tion would represent the estimated effect of the program. 
Well-matched comparison-group studies can be valuable in 
generating hypotheses about what works, but their results need 
to be confirmed in randomized controlled trials for validation 
of cause and effect relationships. 

Randomized Controlled Qroup Study Design. Although the 
most difficult design to implement, there is persuasive evi- 
dence that the randomized controlled group study is superior 
to other evaluation designs in measuring an intervention’s true 
effect. Randomized controlled trials are studies that randomly 
assign individuals to an intervention group or to a control 
group in order to measure the effects of the intervention. For 
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example, in testing whether a mentoring program for third- 
graders is effective in reducing truancy, a large number of 
third-grade students would be randomly assigned to either 
an intervention group, which uses mentoring, or to a control 
group, which does not. The attendance of both groups is then 
measured over time. The difference in attendance between 
the two groups would represent the effect of the mentoring 
program compared to no mentoring program. 

Random assignment of subjects (i.e., students) to groups 
ensures that, as far as possible, there are no initial differences in 
the group memberships systematically influencing and biasing 
outcome measures of the phenomena of interest. The unique 
advantage of random assignment is that it enables a program to 
evaluate whether the intervention itself caused the observed out- 
comes rather than pre-existing differences between the groups in 
other characteristics that affect the outcome. Therefore, the resulting 
difference in outcomes between the intervention and control groups 
can confidently be attributed to the intervention and not to preexist- 
ing differences between the memberships of the two groups. 

A potentially problematic aspect of comparative group studies 
using a true control group, whether assignment of subjects to 
treatment or control groups is randomized or not, is that these 
models require that some potential beneficiaries of the interven- 
tion being assessed will systematically be denied access to it, 
at least for the evaluation period. Rossi, Freeman, and Lipsey 
(1999) point out that 

randomized experiments and other comparative designs can 
only be employed to assess the impacts of partial-coverage 
programs. Partial-coverage programs are those that are to 
be tested on a trial basis or, for whatever reason, are reach- 
ing only a portion of the eligible population. Only in these 
circumstances is it possible to make appropriate comparisons 
between persons who are receiving the intervention and 
comparable persons who are not. (p. 280) 

The practical implications of intentionally omitting from 
participation in an intervention some segment of a population that 
may need the intervention in the interest of rigorous testing can be 
controversial in an actual program situation. For instance, a truancy 
reduction program providing a family involvement intervention to 
parents of half of a group of chronic tmants but no service to parents 
of the other half can intentionally leave children of nonparticipating 
parents at risk of continuing higher tmancy. Planners and evaluators 
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of truancy reduction programs will have to give careful consideration 
to justifications for such a discriminatory practice, the means of selec- 
tion of participants and non-participants, and the special challenges 
that may attend obtaining research data from individuals who are 
not actively participating in the program. One method of addressing 
at least some objections to comparison group designs is to assure that 
nonparticipants in the program intervention during the evaluation 
process, or part of it, will eventually be allowed to be involved in the 
intervention. In this way, participation for control subjects is only 
delayed for a time, not denied indefinitely. 

Phase IV: Instrumentation 

Data Quality 

Selecting and using appropriate evaluation instruments, data 
sources, sampling methods, and multiple measures will help 
ensure that high quality data are collected for the evaluation. 
Several criteria can be used to assess data quality, including valid- 
ity, reliability, accuracy, and utility. Validity is the most important 
consideration. The selected instruments should measure what 
they are supposed to measure. Reliability refers to the consistency 
of assessment results over time or repeated applications. Accuracy 
means that the assessment results are relatively free of measure- 
ment or sampling errors. Finally, data should also be useful and 
user-friendly. It is critical that the data be meaningful to the 
collaborative’s leadership and program staff if they are going to 
use the data for improvement purposes (Yap et ah, 2000). 

Data Sources 

Researchers and evaluators have identified a variety of data 
collection methods including those that measure the processes 
intended to effectively implement the program and those that 
measure the outcomes or results of the program. 

Process Measures. Process measures (sometimes referred to as 
formative measures) are data used to determine if the program 
was implemented as planned. The measures should include a 
description of who is receiving the services, identification of what 
services are being provided, determination of numbers of people 
served by the program, and/or an inventory of the number of vol- 
unteers and staff used to carry out the activities of the program. 
Examples of common process measures used in truancy programs 
include (Justice Research and Statistics Association, 2004): 
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• Number of parental visits to school for school-related events 

• Number of visits to students’ homes by project team members 

• Number of interactions between teachers and youth 

• Number of times project team members reviewed students’ 
homework assignments 

• Number of counseling sessions attended 

In addition, there are a number of other methods for determin- 
ing if the program activities are being implemented in the desired 
way. Such methods include one-on-one interviews of program 
participants, focus group interviews, surveys, on-site observations, 
and case studies. 

Outcome Measures. Outcome measures (sometimes referred to 
as summative measures) are the data used to measure the achieve- 
ment of objectives to determine the effectiveness of the program. 
Examples of common outcome measures used in truancy programs 
include (Justice Research and Statistics Association, 2004): 

• School attendance 

• Academic performance 

• School disciplinary reports 

• Post-school aspirations 

• Arrests (number and type) 

• Adjudications/Convictions (number and type) 

• Substance abuse (alcohol, smoking, and illicit drug use) 

• Fighting at school 

• Bullying at school 

• Family conflict 

Data sources for outcomes measures (and baseline measures of 
the same phenomena) can include individual interviews, focus 
group interviews, surveys, on-site observations, case studies, and 
preexisting data in electronic databases and in paper records and 
documents. 
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Phase V: data collection, analysis, 

SUMMARIZATION, AND INTERPRETATION 
Data Collection Methods 

Pertinent data can be obtained from various sources. Archival 
sources consist of existing documents containing a wide array 
of data such as student assessment data, attendance, and disci- 
pline referrals. Typically, survey and interview data on program 
implementation and outcomes will come from students, teachers, 
administrators, parents, and community members. Behavioral, 
achievement, and attendance data is gathered about students. 
While each data source can provide valuable information, care 
should be taken in deciding which data source may be best for 
which type of information. Using a focus group — which is essen- 
tially a group interview — is more efficient than one-on-one in- 
terviews. However, people often give different answers in groups 
than they do individually. Individuals may feel freer to express 
negative views in a private interview. For both focus groups and 
interviews, the evaluator should have a written interview guide 
that lists the questions and provides space to record answers. 
Good interview questions are open-ended questions written in a 
clear, simple, conversational style. For on-site observations, the 
evaluator needs to develop a guide that describes what he or she 
is looking for. During site visits, evaluators should avoid disrupt- 
ing the activity being observed to ensure valid findings (Yap et 
al, 2000). 

In many instances, it is unlikely that a single measure will suf- 
ficiently assess the achievement of a program objective, especially 
when the objective entails complex knowledge and skills on 
the part of the community, service and justice agencies, schools, 
students, and families. In such cases, the use of multiple measures 
and approaches can enhance the validity, reliability, accuracy, 
and utility of the data as well as decisions about the effectiveness 
of the program activities. Involving program staff and partners in 
designing data collection activities can ensure that the data will 
be used as intended (Yap et al., 2000). 

Data Analysis and Summarization and Interpretation 

Planning and evaluation data are used for two primary types 
of analysis: description and comparison. Descriptive data can be 
used to define the needs for a proposed intervention program and 
to profile community resources, partner capacities, potential ob- 
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Stacies to program success, and other phenomena shaping plans of 
action, organization, and collaboration. The magnitude and char- 
acter of the problem to be addressed can be described through 
reduction and analysis of multiple types of pertinent informa- 
tion. Data representing numbers of truancy events, family court 
actions, recidivism, etc. help provide quantitative descriptions of 
various parameters of truancy in a particular setting. Precision in 
such definitions supports the development of appropriate goals 
which can be assessed via post-intervention measures for evalu- 
ation purposes. Qualitative data from interviews, on-site obser- 
vations, or documents such as court records and police reports 
can be analyzed both qualitatively and quantitatively to provide 
deeper and more nuanced understanding of the complexities of 
the problem and of the requirements for effective intervention in 
specific contexts. 

Comparative uses of data are essential to the program evalu- 
ation process. Initial descriptive data representing the factors 
of interest (arrest rates, parental involvement, confinement 
sentencing rates, etc.) must be established as a basis for compari- 
son with outcome assessments. Although specific criteria such 
as truancy event counts and rates can be assessed during and 
following an intervention program, effectiveness of the program 
cannot be adequately demonstrated, no matter how favorable 
the intermediate or ultimate outcomes, without a pre-interven- 
tion baseline of descriptive data with which to compare outcome 
results. Further, the pre- and post-intervention data have to be in 
compatible formats for effective comparisons to be performed. 

Evaluation research data comprise two basic data types: quan- 
titative and qualitative. Quantitative data is information which is 
inherently numerical. These data reflect phenomena that can be 
counted or measured directly or derived from computations on 
direct counts or measures of phenomena. Qualitative data, on the 
other hand, are not inherently numerical in character but exists 
in the form of words, sounds, and images. However, for planning 
and evaluation purposes qualitative data can be examined and 
summarized through both qualitative and quantitative methods 
of analysis. 

When the information to be analyzed is quantitative, it is usu- 
ally adequate to employ simple summarizations through addition 
or computing of percentage proportions or rates of occurrence. 
Summarization can provide such information as the total number 
of truancies in a specified group of students (ninth grade students 
for instance) per year and for each month of the year and what 
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proportion of these events led to arrest, adjudication, or incar- 
ceration in state schools. These data can be further broken down 
into information related to males and females separately and to 
specific ethnic groups. Similar quantitative data for other high 
school grades can help pinpoint the grades at which truancy is 
most prevalent. Such summarized data can help define the param- 
eters of truancy problems to be addressed in ways that, through 
comparisons with later assessments, inform project managers and 
staff of the success of their ongoing efforts and, potentially, of 
needs to alter intervention programs to improve effectiveness. Fi- 
nally, these data provide a necessary well-defined baseline founda- 
tion for making valid claims of outcome effects for truancy-reduc- 
tion programs. Similarly, data from surveys or interviews in which 
items require discrete responses in the form of ratings, Yes/No 
responses, or reports of numbers of events are readily summarized 
through simple addition or computation of arithmetic averages. 

Qualitative data, e.g., interview responses, can pose problems 
for systematic analysis due to the nature of the information. For 
instance, if program planners interview 100 students and parents, 
or hold 20 focus groups with teachers and school administrators, 
they are typically left with large amounts of open-ended responses 
to questions in the form of written notes or recorded or written 
transcripts of respondents’ statements. It is normal, of course, 
for interviewers to form their own impressions of the important 
points emerging from interviews or other qualitative inqui- 
ries as they go throughout the inquiry process. Flowever, such 
impressions are not rigorously or systematically formulated and, 
although they may be accurate, are not effective in representing 
the problem or factors potentially influencing the potential for 
successful intervention. To bring precision and validity to these 
data, they can be summarized using quantitative methods as well. 
Although complex schemes are often employed for such analyses, 
some involving the use of specialized computer analysis programs, 
rather simple summarizations are usually adequate for providing 
collective descriptions of phenomena of interest to program plan- 
ners or to evaluators. 

Although a transcript of an interview or a focus group discus- 
sion usually contains a large number of statements, certain kinds 
or topics of statements are of particular interest in defining the 
problems of truancy issues for program planning and evaluation. 
For instance, interviews with parents of truants may provide a 
number of expressions of attitudes toward family court judges 
or their decisions. Such statements, though they will have 
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many forms and wide-ranging content, can usually be readily 
interpreted as representing positive or negative attitudes. It is 
fairly straightforward, though time consuming even with a large 
amount of such data, to review notes or transcripts and count 
the total number of relevant statements or references and note 
which are positive and which are negative. From this data the 
percentages of total statements that are positive and negative can 
be easily calculated. This simple analysis provides a proportional 
statistic that can be used in describing one aspect of the truancy 
situation (perhaps identifying a problem area to be addressed 
through aspects of the intervention) and can help identify part- 
ners to involve in formulating a relevant plan of action, support 
meaningful definition of program goals, and be used in evaluating 
project effectiveness through comparison with similar analysis 
of similar data at later times. Such analyses of verbal or other 
qualitative data do not, of course, exhaust the research potentials 
of the data, but typically program planners and developers are 
not seeking exhaustive analysis of data, but targeted investiga- 
tion of specific aspects of the data that are particularly useful in 
identifying and defining problems for programmatic intervention 
and assessment of success. Such simple quantification of complex 
qualitative information directly and adequately addresses these 
purposes, avoids reliance on mere subjective impressions, and en- 
ables precise discussion in writing proposals and program reports. 
Further, such quantified data summaries are more applicable than 
other representations, such as lists of quotations of interviewee 
statements or photographs of graduating classes, in meeting the 
criteria for rigorous evaluation results and reports that funding 
organizations, especially governmental agencies, seek for demon- 
stration of program outcomes. 

Phase VI: Preparation oe the 

EVALUATION REPORT 

E valuation reports detail the interim or final assessments of 
program processes and outcomes. Their purpose is to com- 
municate clearly and comprehensively the findings of evaluation 
inquiries and to provide valid interpretations of the findings to 
stakeholders who have an interest in and use for the results at 
a particular time in the program process. It is the responsibility 
of evaluators to provide clear communications of interim and 
final results in terms that are meaningful and interpretable to the 
target audiences of the reports. Smink and Stank (1992) note, 
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The language in an evaluation report should he clear, concise, 
and free of highly technical terms and educational jargon. 

The data summaries and interpretations should be easily un- 
derstood by readers without any evaluation background. An 
effective evaluation report will be understood by the target 
audiences without additional explanation from the evaluator. 
Decisionmakers should be able to justify program changes, 
continuation, expansion, or termination on the basis of the 
data presented in the evaluation report, (p. 33) 

Evaluation reports can be either formative or summative. 
Formative reports are focused on supporting program effective- 
ness by identifying the ways in which programs are or are not 
meeting their criteria for accomplishment of strategies, objec- 
tives, or goals at various points along the way. This information 
is intended to support informed changes in program activities to 
optimize the probability of ultimate attainment of intended out- 
comes. Summative reports serve the purposes of final assessment 
of overall program success and provide a mechanism for account- 
ability to all stakeholders. 

Smink and Stank (1992) proposed the following steps in 
producing evaluation reports: 

1. Write a brief description of the program, its goals and 
objectives. 

2. List all evaluation questions. 

3. Sort all data summaries and interpretations by evaluation 
questions. 

4. For each evaluation question, write a description of the 
evaluation design, data collection procedures, data analy- 
ses, data summaries and interpretation. 

5. Write the executive summary [and full draft] of the evalua- 
tion report. 

6. Submit a draft of the evaluation report to the project man- 
ager for reaction and approval. 

7. Make revisions and refinements of draft and submit to 
project manager for final approval. 

8. Prepare the final copy of the evaluation report and turn it 
over to the project manager for publication and distribu- 
tion. (p. 34) 
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Presentation of Evaluation Findings 

When summarized data are available for presentation, the 
working spreadsheets, tables, databases, lists, and other formats 
used for analysis and archiving are not always optimal for rep- 
resenting the content of the data in reports and other presenta- 
tions. Narrative summaries and discussions usually are necessary 
to clarify for various audiences what the meanings of the data 
are. This involves interpretation of real situations being reflected 
in the data. Data alone have no meaning. Interpretation by the 
human mind gives information meaning. Interpretation, how- 
ever, is far more susceptible to bias than are measurements and 
computations. Therefore, grounding interpretations of evidence 
of program effects in substantive data that can be traced directly 
back through a chain of analytic processes to original research 
procedures, results, and data sources will strengthen claims made 
for the validity of meanings interpreted from the data. 

Clear representations of data will further enhance claims 
made for validity and accuracy of findings. Often it is useful to 
render summarizations of data analysis in visual form as charts 
or graphs. These representations support both the interpreta- 
tion process and discussion and communication of findings 
and interpretations among partners and outside audiences. For 
example. Table 1 and Figure 1 below present the same informa- 
tion. Flowever, the content is much easier to understand and 
discuss through the visual representation of the graph in Figure 
1 . It is readily apparent from the graph that the data represent 
a declining trend in overall truancies due to the project in 
comparing the first and third year. The graphic presentation also 
enables the viewer to see at a glance that truancies tend to be 
more numerous earlier in the year than later in both Years 1 and 
3 and that School 2 has the greatest number of truancies in all 
time frames. These fictitious examples demonstrate the value of 
visual representation of quantitative data for both analysis and 
presentation. 
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Table 1. Truancy Reports per Quarter for Year 1 and Year 3 of 
Truancy Prevention Project for Four Schools and Total 



Truancy Reports per Quarter 





Year 1 


Year 1 


Year 1 


Year 3 


Year 3 


Year 3 




Aug-Oct 


Nov- Jan 


Feb- April 


Aug-Oct 


Nov-Jan 


Feb- April 


School 1 


25 


15 


10 


8 


5 


3 


School 2 


30 


25 


15 


12 


11 


5 


School 3 


15 


16 


12 


9 


7 


2 


School 4 


17 


18 


9 


6 


6 


1 


Total 


87 


74 


46 


35 


29 


11 



Figure 1. Truancy Reports per Quarter for Year 1 and Year 3 of 
Truancy Prevention Project for Four Schools and Total 




Novemb«r> 

January 


February*April 


August -October 


November* 

January 


February-April 


Year 1 


Year 1 


Year 3 


Year 3 


Years 



Year and Quarter of Truancy Project 



In summary, the use of straightforward and simple treatments 
of data, both quantitative and qualitative, can strengthen infor- 
mational foundations for truancy program planning, assessment 
of program progress, interpretation of the value of interventions, 
and determination of the effectiveness and ultimate outcomes of 
truancy prevention programs. 
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Conclusion 



A n effective truancy reduction program — just as any 
program designed to promote systemic change to 
improve the conditions of schools, youth, families, and 
communities — is dependent on careful attention to three stages: 
planning, implementation, and evaluation. Evaluation should be 
an integral part of the effort from the beginning. Evaluation data 
can be used to guide program improvement by measuring the 
program’s progress in achieving its goals and can be used to refine 
or alter programs or services. Strategies to reduce truancy require 
a major investment of resources and energy. Evaluation data 
provides evidence to funding agencies, policymakers, and other 
stakeholders that their investments are producing better out- 
comes for youth, families, and communities. Evaluation results 
can account for how resources were used and document program 
effectiveness to justify the costs of such efforts. 
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TOOLS 



T he Tools Section of this publication contains three 

tools designed to facilitate the planning and evaluation 
of a truancy reduction program: ( 1 ) Logic Model for 
the Evaluation of Results-Based Truancy Reduction Programs: 
Example and Worksheets; (2) Creating an Effective Truancy 
Reduction Program Design and Evaluation: Guidelines and 
Worksheets; and (3) Developing an Action and Evaluation Plan 
for a Truancy Reduction Program: Guidelines and Worksheets. 
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LOGIC Model for the Evaluation of 
ResultS'Based Truancy Reduction 
Programs 

Example and Worksheets 



Principles Guiding the Work 

• Diversity is valued 

• Families are actively involved 

• Youth are treated with respect 

• Collaboration within the community is critical 




Colorado Foundation for Families and Children. (2005). The logic model for the evaluation of the truancy 
program. Denver, CO: Colorado Foundation for Families and Children. 

Available online: http://www.schoolengagement.org/TruancypreventionRegistry/Admin/Resources/ 
Resources/2 1 .pdf 
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Evaluation 



N eed/Capacity 


Strategy 


Milestones 




Family: 




Referral to 




More employment, 




Employment, SES, 




community 




access to childcare, 




transportation, 




resources 




increased educa- 




childcare 








tional involve- 


Results 










ment 














School: 




Educate teachers 




Safety, reduced 






After-school pro- 




about outcomes of 




teacher turnover, 






grams, tutoring, 




truancy 




more services 






counseling 












Youth are in 














school and 








out of Juvenile 


Student: 




Mentoring, 




Improved 




Justice system 


Academic 




tutoring 




attendance. 






abilities, social. 








grades, behavior 






behavioral, peers. 














Juvenile Justice 






















Community: 




Education, media. 




Community 






Recreation, 




outreach 




involvement with 




agency collabora- 








youth, resources 




tion, faith-based 








accessed, etc. 




organizations 




















Process Evaluation 




Outcome Evaluation 



Colorado Foundation for Families and Children. (2005). The logic model for the evaluation of the truancy prO' 
gram. Denver, CO: Colorado Foundation for Families and Children. 

Available online; http://www.schoolengagement.org/TruancypreventionRegistry/Admin/Resources/ 
Resources/21. pdf 
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Example Worksheet 



Need/Capacity Strategy Milestones 



Target 

Families 



Service 

Providers 



Target Children 



Target 

Community 

Factors 



Results 



Process Evaluation 



Outcome Evaluation 



Colorado Foundation for Families and Children. (2005). The logic model for the evaluation of the truancy 
program. Denver, CO: Colorado Foundation for Families and Children. 

Available online: http://www.schoolengagement.org/TruancypreventionRegistry/Admin/Resources/ 
Resources/21 .pdf 
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Creating an Effective Truancy 
Reduction Program Design and 
Evaluation 

Guidelines 



A program evaluation is a comprehensive and systematic 
assessment of the extent to which program goals and objectives 
are being achieved as well as the effectiveness of the strategies 
and services implemented to achieve them. Designing an effeC' 
tive program evaluation involves the following: 

• Planning and implementing a data collection process 

• Determining and using appropriate data analyses 

• Planning and preparing evaluation reports 

• Using the evaluation results to improve the program 
What is your data collection process? 

A data collection process is a set of formal procedures and 
tools used for gathering information required to answer evalu- 
ation questions. Planning an effective data collection process 
involves: 

• Determining what data must be collected 

• Determining where data are located 

• Deciding how to collect the data 

• Planning when to collect the data 

• Deciding who will collect the data 

WHAT data should be collected^ Determining what data 
should be collected is an essential first step in designing the 
data collection process. Data collection procedures can be 
time-consuming and expensive. Thus, when planning your 
evaluation try to resist the temptation of collecting unneeded 
information. If you have clearly written, well-defined objec- 
tives and strategies this important step will not be too difficult. 
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WHERE are the data located? The second step in planning 
your data collection process is to determine where the neces- 
sary data are located. In some cases the data may already exist 
and will simply need to he collected. In other cases, the data 
must be generated (e.g., questionnaires, surveys, observations, 
interviews, etc.) If completed during the initial planning stage 
of your program, this exercise may alert you to the need for ad- 
ditional records. 

HOW should data he collected? Many different kinds of 
methods may be used to collect data ranging from relatively 
simple reviews of program records to complex assessments of 
knowledge, skills, or attitudes. Data collection methods may 
include procedures such as the following: 

• Reviewing program records 

• Selecting/developing and conducting interviews 

• Selecting/developing and conducting surveys 

• Selecting/developing measures of affective characteristics 

• Selecting/developing measures of cognitive knowledge 

• Selecting/developing observation methods 

WHEN should data he collected? Making decisions about 
when to collect data is very important. As much as possible, 
data collection procedures should be integrated into regular 
program activities. In this way, your evaluation effort will be 
viewed not as a separate function but as part of normal program 
operations. 

WHO should collect the data? While you may have one 
person (e.g., executive director or external evaluator) coordi- 
nating overall data collection activities, many people may be 
involved in actually collecting data. The answers to the follow- 
ing questions may help when deciding who should collect data: 

• How often will data need to be collected? 

• How much time will it take? 

• Who has access to the data? 

• Are any special skills needed to collect the data? 



Creating an Effective Truancy Reduction Program Design and Evaluation has been 
adapted from an evaluation handbook developed cooperatively by the National 
Dropout Prevention Center and South Carolina Communities In Schools for program 
evaluation of the Communities In Schools after-school initiative. 
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Worksheets 



What is your vision for your program? 

A vision is a global statement that describes the overall purpose 
of a program. 

An effective vision statement: 

• Is a positive statement written in the present tense 

• Describes proactive efforts rather than reactive efforts 

• Focuses on anticipated benefits and accomplishments 

• Applies to all young people and families targeted for par- 
ticipation 

• Reflects expectations of excellence 

• Communicates a true collaborative effort and spirit 

Write your project vision statement 
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What are the goals of your program? 

Program goals are broad statements that describe what must be 
accomplished to achieve your vision. 

Write your project goals. 
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What are your program objectives? 

Program objectives are specific statements describing measur- 
able outcomes related to program goals. A well-written program 
objective includes the following: 

• Describes a desired outcome rather than a strategy for 
achieving the outcome 

• Is clearly stated in measurable terms that 

• defines who will change (e.g., program participants) 

• defines what will change (e.g., knowledge, skills, 
attitudes, and behavior) 

• defines when change will occur (e.g., by the end of each 
grading period or by the end of the school year) 

Write your project objectives. 

Goal # 

Objective # 



Goal # 
Objective # 



Goal # 
Objective # 
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What are your program strategies? 

Program strategies are specific actions, core services, or events 
designed to accomplish one or more program objectives. 

Objectives and strategies for project goal # 



Objectives 


Strategies 
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Developing an action and Evaluation Plan 
LORA Truancy Reduction Program 

Guidelines and Worksheets 



Worksheet #1: Action Plan 



Goal 


Objective 


Resources 


Persons 


Timeline 






Needed 


Responsible 


Starting date, interim dates, 
completion date 



Worksheet #1 has five columns to fill out 
Goal - Insert the goal. 

Objective - list each major objective separately (what you will do to accomplish the 
goal). You may want to print multiple copies of Worksheet #1. One or more pages 
of the worksheet can be devoted to the objectives for one goal. 

Resources Needed - Identify the persons who will be responsible for the objective 
and that the resources for the objective are secured. 

Timeline - Identify the date that the objective will be started, interim target dates, 
and when the objective will be completely implemented. 



These Guidelines were developed by the National Dropout Prevention Center for use with the New York State 
Program Assessment and Review (PAR). 
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Worksheet #1: Action Plan 



Goal 

# 


Objective 


Resources 

Needed 


Persons 

Responsible 


Timeline 

Starting date , interim dates , 
completion date 
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Worksheet #2: Evaluation Plan 



Expected Outcome 


Activity 


What data 


Who will collect it? 


When? 


(objective) 




will be collected? 






Who? What? By how 










much? By when? 











Worksheet #2 has five columns to fill out 

Expected Outcome - In this column list the outcomes or changes you expect 
(e.g., who? The affected group; What? Behaviors that are expected to change; 

By how much? The percentage or rate of change or improvement; By when? 

The date or period by which you expect the change). 

Activity - What strategy will the program use to achieve the expected outcome? 

What data will be collected? - see worksheet #3 to identify assessment methods and 
data sources you may want to consider. 

Who will collect it? - Identify the person or persons who will be responsible for 
collecting the data. 

When - When does the baseline data need to be collected against which you will 
measure your program’s progress? When do the outcome types of data need to be 
collected, surveys given, observations completed, etc. 
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Worksheet #2: Evaluation Plan 



Expected Outcome 
(objective) 

Who? What? By how 
much? By when? 


Activity 


What data 
will be collected? 


Who will collect it? 


When? 
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Worksheet #3: Program Assessment Methods and Data Sources 



List each expected outcome and the activities designed to accomplish the outcome. 
Decide the logical methods for determining if the outcome has been achieved. 
What sources of data can you use to show that each activity is effective? 



Expected 
Outcome # 


Assessment Methods/Data Sources 


Activity 


Surveys 


Interviews 


Observations 


Discipline 

referrals 


Attendance 


Test Scores 


Participation 
in School 
Activities 


Social 

Services 

Report 


Juvenile 
Justice Record 


Other 
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